- Deep learning
- Feed-forward neural networks
- Recurrent neural networks
A machine learning subfield of learning representations of data. Exceptional effective at learning patterns.
Deep learning algorithms attempt to learn (multiple levels of) representation by using a hierarchy of multiple layers.
\[h = \sigma(W_1x + b_1)\] \[y = \sigma(W_2h + b_2)\]
Learned hypothesis may fit the training data very well, even outliers ( noise) but fail to generalize to new examples (test data)
How to avoid overfitting?
Suppose we had the following scenario:
Day 1: Lift Weights
Day 2: Swimming
Day 3: At this point, our model must decide whether we should take a rest day or yoga. Unfortunately, it only has access to the previous day. In other words, it knows we swam yesterday but it doesn’t know whether had taken a break the day before. Therefore, it can end up predicting yoga.